# Distributed Computing

DeepSeek Profile Data is a project focused on performance analysis of deep learning frameworks. It captures performance data of training and inference frameworks through PyTorch Profiler, helping researchers and developers better understand computation and communication overlap strategies and underlying implementation details. This data is crucial for optimizing large-scale distributed training and inference tasks, significantly improving system efficiency and performance. This project is a significant contribution from the DeepSeek team in the field of deep learning infrastructure, aiming to promote the community's exploration of efficient computing strategies.

Exo is an experimental software project designed to harness existing household devices such as iPhones, iPads, Androids, Macs, and Linux to unite them as a powerful GPU to run AI models. It supports a variety of popular models such as LLaMA and features dynamic model partitioning that optimally divides models based on the current network topology and device resources. Additionally, Exo provides APIs compatible with ChatGPT, allowing the use of Exo to run models in applications with just a simple line of code change.

Video-Infinity is a distributed long video generation technology that can generate 2300 frames of video in 5 minutes, which is 100 times faster than previous methods. Based on the VideoCrafter2 model, it adopts innovative technologies such as Clip Parallelism and Dual-scope Attention, significantly improving the efficiency and quality of video generation.

AsyncDiff is a method for accelerating diffusion models through asynchronous denoising parallelization. It divides the noise prediction model into multiple components and distributes them across different devices, enabling parallel processing. This approach significantly reduces inference latency while having a minimal impact on generation quality. AsyncDiff supports a variety of diffusion models, including Stable Diffusion 2.1, Stable Diffusion 1.5, Stable Diffusion x4 Upscaler, Stable Diffusion XL 1.0, ControlNet, Stable Video Diffusion, and AnimateDiff.

YaFSDP is a distributed data parallelism framework designed to work well with transformer-like neural network architectures. It is 20% faster than the traditional FSDP when pre-training large language models (LLMs) and performs better under high-memory pressure conditions. YaFSDP aims to reduce the overhead of communication and memory operations.